Medical subdomain classification of clinical notes using a machine learning-based natural language processing approach.

BACKGROUND: The medical subdomain of a clinical note, such as cardiology or neurology, is useful content-derived metadata for developing machine learning downstream applications. To classify the medical subdomain of a note accurately, we have constructed a machine learning-based natural language processing (NLP) pipeline and developed medical subdomain classifiers based on the content of the note. METHODS: We constructed the pipeline using the clinical NLP system, clinical Text Analysis and Knowledge Extraction System (cTAKES), the Unified Medical Language System (UMLS) Metathesaurus, Semantic Network, and learning algorithms to extract features from two datasets - clinical notes from Integrating Data for Analysis, Anonymization, and Sharing (iDASH) data repository (n = 431) and Massachusetts General Hospital (MGH) (n = 91,237), and built medical subdomain classifiers with different combinations of data representation methods and supervised learning algorithms. We evaluated the performance of classifiers and their portability across the two datasets. RESULTS: The convolutional recurrent neural network with neural word embeddings trained-medical subdomain classifier yielded the best performance measurement on iDASH and MGH datasets with area under receiver operating characteristic curve (AUC) of 0.975 and 0.991, and F1 scores of 0.845 and 0.870, respectively. Considering better clinical interpretability, linear support vector machine-trained medical subdomain classifier using hybrid bag-of-words and clinically relevant UMLS concepts as the feature representation, with term frequency-inverse document frequency (tf-idf)-weighting, outperformed other shallow learning classifiers on iDASH and MGH datasets with AUC of 0.957 and 0.964, and F1 scores of 0.932 and 0.934 respectively. We trained classifiers on one dataset, applied to the other dataset and yielded the threshold of F1 score of 0.7 in classifiers for half of the medical subdomains we studied. CONCLUSION: Our study shows that a supervised learning-based NLP approach is useful to develop medical subdomain classifiers. The deep learning algorithm with distributed word representation yields better performance yet shallow learning algorithms with the word and concept representation achieves comparable performance with better clinical interpretability. Portable classifiers may also be used across datasets from different institutions.

Non-visit-based cancer screening using a novel population management system.

BACKGROUND: Advances in information technology (IT) now permit population-based preventive screening, but the best methods remain uncertain. We evaluated whether involving primary care providers (PCPs) in a visit-independent population management IT application led to more effective cancer screening. METHODS: We conducted a cluster-randomized trial involving 18 primary care practice sites and 169 PCPs from June 15, 2011, to June 14, 2012. Participants included adults eligible for breast, cervical, and/or colorectal cancer screening. In practices randomized to the intervention group, PCPs reviewed real-time rosters of their patients overdue for screening and provided individualized contact (via a letter, practice delegate, or patient navigator) or deferred screening (temporarily or permanently). In practices randomized to the comparison group, overdue patients were automatically sent reminder letters and transferred to practice delegate lists for follow-up. Intervention patients without PCP action within 8 weeks defaulted to the automated control version. The primary outcome was adjusted average cancer screening completion rates over 1-year follow-up, accounting for clustering by physician or practice. RESULTS: Baseline cancer screening rates (80.8% vs 80.3%) were similar among patients in the intervention (n = 51,071) and comparison group (n = 52,799). Most intervention providers used the IT application (88 of 101, 87%) and users reviewed 7984 patients overdue for at least 1 cancer screening (73% sent reminder letter, 6% referred directly to a practice delegate or patient navigator, and 21% deferred screening). In addition, 6128 letters were automatically sent to patients in the intervention group (total of 12,002 letters vs 16,378 letters in comparison practices; P < .001). Adjusted average cancer screening rates did not differ among intervention and comparison practices for all cancers combined (81.6% vs 81.4%; P = .84) nor breast (82.7% vs 82.7%; P = .96), cervical (84.1% vs 84.7%; P = .60), or colorectal cancer (77.8% vs 76.2%; P = .33). CONCLUSIONS: Involving PCPs in a visit-independent population management IT application resulted in similar cancer screening rates compared with an automated reminder system, but fewer patients were sent reminder letters. This suggests that PCPs were able to identify and exclude from contact patients who would have received automated reminder letters but not undergone screening.

Development and implementation of a real-time 30-day readmission predictive model.

Hospitals are under great pressure to reduce readmissions of patients. Being able to reliably predict patients at increased risk for rehospitalization would allow for tailored interventions to be offered to them. This requires the creation of a functional predictive model specifically designed to support real-time clinical operations. A predictive model for readmissions within 30 days of discharge was developed using retrospective data from 45,924 MGH admissions between 2/1/2012 and 1/31/2013 only including factors that would be available by the day after admission. It was then validated prospectively in a real-time implementation for 3,074 MGH admissions between 10/1/2013 and 10/31/2013. The model developed retrospectively had an AUC of 0.705 with good calibration. The real-time implementation had an AUC of 0.671 although the model was overestimating readmission risk. A moderately discriminative real-time 30-day readmission predictive model can be developed and implemented in a large academic hospital.

Applying operations research to optimize a novel population management system for cancer screening.

OBJECTIVE: To optimize a new visit-independent, population-based cancer screening system (TopCare) by using operations research techniques to simulate changes in patient outreach staffing levels (delegates, navigators), modifications to user workflow within the information technology (IT) system, and changes in cancer screening recommendations. MATERIALS AND METHODS: TopCare was modeled as a multiserver, multiphase queueing system. Simulation experiments implemented the queueing network model following a next-event time-advance mechanism, in which systematic adjustments were made to staffing levels, IT workflow settings, and cancer screening frequency in order to assess their impact on overdue screenings per patient. RESULTS: TopCare reduced the average number of overdue screenings per patient from 1.17 at inception to 0.86 during simulation to 0.23 at steady state. Increases in the workforce improved the effectiveness of TopCare. In particular, increasing the delegate or navigator staff level by one person improved screening completion rates by 1.3% or 12.2%, respectively. In contrast, changes in the amount of time a patient entry stays on delegate and navigator lists had little impact on overdue screenings. Finally, lengthening the screening interval increased efficiency within TopCare by decreasing overdue screenings at the patient level, resulting in a smaller number of overdue patients needing delegates for screening and a higher fraction of screenings completed by delegates. CONCLUSIONS: Simulating the impact of changes in staffing, system parameters, and clinical inputs on the effectiveness and efficiency of care can inform the allocation of limited resources in population management.

Assessing hospital readmission risk factors in heart failure patients enrolled in a telemonitoring program.

The purpose of this study was to validate a previously developed heart failure readmission predictive algorithm based on psychosocial factors, develop a new model based on patient-reported symptoms from a telemonitoring program, and assess the impact of weight fluctuations and other factors on hospital readmission. Clinical, demographic, and telemonitoring data was collected from 100 patients enrolled in the Partners Connected Cardiac Care Program between July 2008 and November 2011. 38% of study participants were readmitted to the hospital within 30 days. Ten different heart-failure-related symptoms were reported 17,389 times, with the top three contributing approximately 50% of the volume. The psychosocial readmission model yielded an AUC of 0.67, along with sensitivity 0.87, specificity 0.32, positive predictive value 0.44, and negative predictive value 0.8 at a cutoff value of 0.30. In summary, hospital readmission models based on psychosocial characteristics, standardized changes in weight, or patient-reported symptoms can be developed and validated in heart failure patients participating in an institutional telemonitoring program. However, more robust models will need to be developed that use a comprehensive set of factors in order to have a significant impact on population health.

Efficacy of a clinical decision-support system in an HIV practice: a randomized trial.

BACKGROUND: Data to support improved patient outcomes from clinical decision-support systems (CDSSs) are lacking in HIV care. OBJECTIVE: To test the efficacy of a CDSS in improving HIV outcomes in an outpatient clinic. DESIGN: Randomized, controlled trial. (ClinicalTrials.gov registration number: NCT00678600) SETTING: Massachusetts General Hospital HIV Clinic. PARTICIPANTS: HIV care providers and their patients. INTERVENTION: Computer alerts were generated for virologic failure (HIV RNA level >400 copies/mL after a previous HIV RNA level ≤400 copies/mL), evidence of suboptimal follow-up, and 11 abnormal laboratory test results. Providers received interactive computer alerts, facilitating appointment rescheduling and repeated laboratory testing, for half of their patients and static alerts for the other half. MEASUREMENTS: The primary end point was change in CD4 cell count. Other end points included time to clinical event, 6-month suboptimal follow-up, and severe laboratory toxicity. RESULTS: Thirty-three HIV care providers followed 1011 patients with HIV. In the intervention group, the mean increase in CD4 cell count was greater (0.0053 vs. 0.0032 × 109 cells/L per month; difference, 0.0021 × 109 cells/L per month [95% CI, 0.0001 to 0.004]; P = 0.040) and the rate of 6-month suboptimal follow-up was lower (20.6 vs. 30.1 events per 100 patient-years; P = 0.022) than those in the control group. Median time to next scheduled appointment was shorter in the intervention group than in the control group after a suboptimal follow-up alert (1.71 vs. 3.48 months; P < 0.001) and after a toxicity alert (2.79 vs. >6 months; P = 0.072). More than 90% of providers supported adopting the CDSS as part of standard care. LIMITATION: This was a 1-year informatics study conducted at a single hospital subspecialty clinic. CONCLUSION: A CDSS using interactive provider alerts improved CD4 cell counts and clinic follow-up for patients with HIV. Wider implementation of such systems can provide important clinical benefits. PRIMARY FUNDING SOURCE: National Institute of Allergy and Infectious Diseases.

Impact of a computer-based diagnostic decision support tool on the differential diagnoses of medicine residents.

BACKGROUND: Computer-based medical diagnostic decision support systems have been used for decades, initially as stand-alone applications. More recent versions have been tested for their effectiveness in enhancing the diagnostic ability of clinicians. OBJECTIVE: To determine if viewing a rank-ordered list of diagnostic possibilities from a medical diagnostic decision support system improves residents' differential diagnoses or management plans. METHOD: Twenty first-year internal medicine residents at Massachusetts General Hospital viewed 3 deidentified case descriptions of real patients. All residents completed a web-based questionnaire, entering the differential diagnosis and management plan before and after seeing the diagnostic decision support system's suggested list of diseases. In all 3 exercises, the actual case diagnosis was first on the system's list. Each resident served as his or her own control (pretest/posttest). RESULTS: For all 3 cases, a substantial percentage of residents changed their primary considered diagnosis after reviewing the system's suggested diagnoses, and a number of residents who had not initially listed a "further action" (laboratory test, imaging study, or referral) added or changed their management options after using the system. Many residents (20% to 65% depending on the case) improved their differential diagnosis from before to after viewing the system's suggestions. The average time to complete all 3 cases was 15.4 minutes. Most residents thought that viewing the medical diagnostic decision support system's list of suggestions was helpful. CONCLUSION: Viewing a rank-ordered list of diagnostic possibilities from a diagnostic decision support tool had a significant beneficial effect on the quality of first-year medicine residents' differential diagnoses and management plans.

Linking electronic health record-extracted psychosocial data in real-time to risk of readmission for heart failure.

BACKGROUND: Knowledge of psychosocial characteristics that helps to identify patients at increased risk for readmission for heart failure (HF) may facilitate timely and targeted care. OBJECTIVE: We hypothesized that certain psychosocial characteristics extracted from the electronic health record (EHR) would be associated with an increased risk for hospital readmission within the next 30 days. METHODS: We identified 15 psychosocial predictors of readmission. Eleven of these were extracted from the EHR (six from structured data sources and five from unstructured clinical notes). We then analyzed their association with the likelihood of hospital readmission within the next 30 days among 729 patients admitted for HF. Finally, we developed a multivariable predictive model to recognize individuals at high risk for readmission. RESULTS: We found five characteristics-dementia, depression, adherence, declining/refusal of services, and missed clinical appointments-that were associated with an increased risk for hospital readmission: the first four features were captured from unstructured clinical notes, while the last item was captured from a structured data source. CONCLUSIONS: Unstructured clinical notes contain important knowledge on the relationship between psychosocial risk factors and an increased risk of readmission for HF that would otherwise have been missed if only structured data were considered. Gathering this EHR-based knowledge can be automated, thus enabling timely and targeted care.

A cluster-randomized trial of a primary care informatics-based system for breast cancer screening.

BACKGROUND: Information technology offers the promise, as yet unfulfilled, of delivering efficient, evidence-based health care. OBJECTIVE: To evaluate whether a primary care network-based informatics intervention can improve breast cancer screening rates. DESIGN: Cluster-randomized controlled trial of 12 primary care practices conducted from March 20, 2007 to March 19, 2008. PATIENTS: Women 42-69 years old with no record of a mammogram in the prior 2 years. INTERVENTIONS: In intervention practices, a population-based informatics system was implemented that: connected overdue patients to appropriate care providers, presented providers with a Web-based list of their overdue patients in a non-visit-based setting, and enabled "one-click" mammography ordering or documented deferral reasons. Patients selected for mammography received automatically generated letters and follow-up phone calls. All practices had electronic health record reminders about breast cancer screening available during clinical encounters. MAIN MEASURES: The primary outcome was the proportion of overdue women undergoing mammography at 1-year follow-up. KEY RESULTS: Baseline mammography rates in intervention and control practices did not differ (79.5% vs 79.3%, p = 0.73). Among 3,054 women in intervention practices and 3,676 women in control practices overdue for mammograms, intervention patients were somewhat younger, more likely to be non-Hispanic white, and have health insurance. Most intervention providers used the system (65 of 70 providers, 92.9%). Action was taken for 2,652 (86.8%) intervention patients [2,274 (74.5%) contacted and 378 (12.4%) deferred]. After 1 year, mammography rates were significantly higher in the intervention arm (31.4% vs 23.3% in control arm, p < 0.001 after adjustment for baseline differences; 8.1% absolute difference, 95% CI 5.1-11.2%). All demographic subgroups benefited from the intervention. Intervention patients completed screening sooner than control patients (p < 0.001). CONCLUSIONS: A novel population-based informatics system functioning as part of a non-visit-based care model increased mammography screening rates in intervention practices. TRIAL REGISTRATION: ClinicalTrials.gov; NCT00462891.

A Clinical Database-Driven Approach to Decision Support: Predicting Mortality Among Patients with Acute Kidney Injury.

In exploring an approach to decision support based on information extracted from a clinical database, we developed mortality prediction models of intensive care unit (ICU) patients who had acute kidney injury (AKI) and compared them against the Simplified Acute Physiology Score (SAPS). We used MIMIC, a public de-identified database of ICU patients admitted to Beth Israel Deaconess Medical Center, and identified 1400 patients with an ICD9 diagnosis of AKI and who had an ICU stay > 3 days. Multivariate regression models were built using the SAPS variables from the first 72 hours of ICU admission. All the models developed on the training set performed better than SAPS (AUC = 0.64, Hosmer-Lemeshow p < 0.001) on an unseen test set; the best model had an AUC = 0.74 and Hosmer-Lemeshow p = 0.53. These findings suggest that local customized modeling might provide more accurate predictions. This could be the first step towards an envisioned individualized point-of-care probabilistic modeling using one's clinical database.

Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2).

Informatics for Integrating Biology and the Bedside (i2b2) is one of seven projects sponsored by the NIH Roadmap National Centers for Biomedical Computing (http://www.ncbcs.org). Its mission is to provide clinical investigators with the tools necessary to integrate medical record and clinical research data in the genomics age, a software suite to construct and integrate the modern clinical research chart. i2b2 software may be used by an enterprise's research community to find sets of interesting patients from electronic patient medical record data, while preserving patient privacy through a query tool interface. Project-specific mini-databases ("data marts") can be created from these sets to make highly detailed data available on these specific patients to the investigators on the i2b2 platform, as reviewed and restricted by the Institutional Review Board. The current version of this software has been released into the public domain and is available at the URL: http://www.i2b2.org/software.

Queuing theory to guide the implementation of a heart failure inpatient registry program.

OBJECTIVE The authors previously implemented an electronic heart failure registry at a large academic hospital to identify heart failure patients and to connect these patients with appropriate discharge services. Despite significant improvements in patient identification and connection rates, time to connection remained high, with an average delay of 3.2 days from the time patients were admitted to the time connections were made. Our objective for this current study was to determine the most effective solution to minimize time to connection. DESIGN We used a queuing theory model to simulate 3 different potential solutions to decrease the delay from patient identification to connection with discharge services. MEASUREMENTS The measures included average rate at which patients were being connected to the post discharge heart failure services program, average number of patients in line, and average patient waiting time. RESULTS Using queuing theory model simulations, we were able to estimate for our current system the minimum rate at which patients need to be connected (262 patients/mo), the ideal patient arrival rate (174 patients/mo) and the maximal patient arrival rate that could be achieved by adding 1 extra nurse (348 patients/mo). CONCLUSIONS Our modeling approach was instrumental in helping us characterize key process parameters and estimate the impact of adding staff on the time between identifying patients with heart failure and connecting them with appropriate discharge services.

A rheumatology-specific informatics-based application with a disease activity calculator.

OBJECTIVE: To design a rheumatology-specific tool with a disease activity calculator integrated into the electronic medical records (EMRs) at our institution and assess physicians' attitudes toward the use of this tool. METHODS: The Rheumatology OnCall (ROC) application culls rheumatology-pertinent data from our institution's laboratory, microbiology, pathology, radiology, and pharmacy information systems. Attending rheumatologists and rheumatology fellows accessed the ROC and disease activity calculator during outpatient visits at the time of the clinical encounter. RESULTS: During the 12-week study period, 15 physicians used the ROC application and the disease activity calculator during 474 and 429 outpatient clinic visits, respectively. In weekly survey responses, physicians reported that use of the ROC interface improved patient care in 140 (78%) of 179 visits, and that the Disease Activity Score in 28 joints (DAS28) results at the time of the visit would not have changed patient management in 157 (88%) of these, although seeing a trend in DAS28 was useful in 149 (96%) of 156 visits. At the study's conclusion, most physicians reported that the ROC application was useful (11 of 12 physicians) and that seeing a trend in DAS28 improved daily patient care (12 of 13 physicians). CONCLUSION: The ROC application is useful in daily rheumatologic care, and the disease activity calculator facilitates management of patients with rheumatoid arthritis. However, widespread acceptance and use of such tools depend upon the general acceptance of and access to EMRs in the clinical setting. The utility of the disease activity calculator may be limited by the lack of available acute-phase reactant results at the time of the clinical encounter.

Physician ability to assess rheumatoid arthritis disease activity using an electronic medical record-based disease activity calculator.

OBJECTIVE: To assess physicians' concordance with Disease Activity Score in 28 joints (DAS28) categories calculated by an electronic medical record (EMR)-embedded disease activity calculator, as well as attitudes toward this application. METHODS: Fifteen rheumatologists used the EMR-embedded disease activity calculator to predict a rheumatoid arthritis (RA) DAS28 disease activity category at the time of each clinical encounter. RESULTS: Physician-predicted DAS28 disease activity categories ranged from high (>5.1, 15% of cohort, 66 of 429 patient visits) to moderate (>3.2-5.1, 21% of cohort, 90 of 429 patient visits) to low (2.6-3.2, 29% of cohort, 123 of 429 patient visits) to remission (<2.6, 35% of cohort, 150 of 429 patient visits). Overall concordance between calculated DAS28 results and physician-predicted RA disease activity was 64%. Using either the physician-predicted or the calculated DAS28 category as the gold standard, accuracy was greatest for patients in remission (75% and 88% accuracy, respectively) and those with high disease activity (68% and 79% accuracy, respectively), and less for patients with moderate (48% and 62% accuracy, respectively) or low disease activity (62% and 31% accuracy, respectively). CONCLUSION: Accurate physician prediction of DAS28 remission and high disease activity categories, even without immediate availability of the erythrocyte sedimentation rate or the C-reactive protein level at the time of the visit, may be used to guide quantitatively driven outpatient RA management.

Mammography FastTrack: an intervention to facilitate reminders for breast cancer screening across a heterogeneous multi-clinic primary care network.

Health care information technology can be a means to improve quality and efficiency in the primary care setting. However, merely applying technology without addressing how it fits into provider workflow and existing systems is unlikely to achieve improvement goals. Improving quality of primary care, such as cancer screening rates, requires addressing barriers at system, provider, and patient levels. The authors report the development, implementation, and preliminary use of a new breast cancer screening outreach program in a large multicenter primary care network. This installation paired population-based surveillance with customized information delivery based on a validated model linking patients to providers and practices. In the first six months, 86% of physicians and all case managers voluntarily participated in the program. Providers intervened in 83% of the mammogram-overdue population by initiating mailed reminders or deferring contact. Overall, 63% of patients were successfully contacted. Systematic population-based efforts are promising tools to improve preventative care.

Designing healthcare information technology to catalyse change in clinical care.

The gap between best practice and actual patient care continues to be a pervasive problem in our healthcare system. Efforts to improve on this knowledge-performance gap have included computerised disease management programs designed to improve guideline adherence. However, current computerised reminder and decision support interventions directed at changing physician behaviour have had only a limited and variable effect on clinical outcomes. Further, immediate pay-for-performance financial pressures on institutions have created an environment where disease management systems are often created under duress, appended to existing clinical systems and poorly integrated into the existing workflow, potentially limiting their real-world effectiveness. The authors present a review of disease management as well as a conceptual framework to guide the development of more effective health information technology (HIT) tools for translating clinical information into clinical action.

Lessons from implementing a combined workflow-informatics system for diabetes management.

Shortcomings surrounding the care of patients with diabetes have been attributed largely to a fragmented, disorganized, and duplicative health care system that focuses more on acute conditions and complications than on managing chronic disease. To address these shortcomings, we developed a diabetes registry population management application to change the way our staff manages patients with diabetes. Use of this new application has helped us coordinate the responsibilities for intervening and monitoring patients in the registry among different users. Our experiences using this combined workflow-informatics intervention system suggest that integrating a chronic disease registry into clinical workflow for the treatment of chronic conditions creates a useful and efficient tool for managing disease.

Diabetes information technology: designing informatics systems to catalyze change in clinical care.

Current computerized reminder and decision support systems intended to improve diabetes care have had a limited effect on clinical outcomes. Increasing pressures on health care networks to meet standards of diabetes care have created an environment where information technology systems for diabetes management are often created under duress, appended to existing clinical systems, and poorly integrated into the existing workflow. After defining the components of diabetes disease management, the authors present an eight-step conceptual framework to guide the development of more effective diabetes information technology systems for translating clinical information into clinical action.

A self-scaling, distributed information architecture for public health, research, and clinical care.

OBJECTIVE: This study sought to define a scalable architecture to support the National Health Information Network (NHIN). This architecture must concurrently support a wide range of public health, research, and clinical care activities. STUDY DESIGN: The architecture fulfils five desiderata: (1) adopt a distributed approach to data storage to protect privacy, (2) enable strong institutional autonomy to engender participation, (3) provide oversight and transparency to ensure patient trust, (4) allow variable levels of access according to investigator needs and institutional policies, (5) define a self-scaling architecture that encourages voluntary regional collaborations that coalesce to form a nationwide network. RESULTS: Our model has been validated by a large-scale, multi-institution study involving seven medical centers for cancer research. It is the basis of one of four open architectures developed under funding from the Office of the National Coordinator of Health Information Technology, fulfilling the biosurveillance use case defined by the American Health Information Community. The model supports broad applicability for regional and national clinical information exchanges. CONCLUSIONS: This model shows the feasibility of an architecture wherein the requirements of care providers, investigators, and public health authorities are served by a distributed model that grants autonomy, protects privacy, and promotes participation.